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A number  of  models  and  corresponding  replacement  policies  have 
been  developed  for  stochastically  deteriorating  systems  In  an  effort 
to  reduce  system  operation  and  maintenance  costs.  The  focus  of  this 
project  has  been  directed  toward  the  class  of  seml-Markovlan  replace- 
sent  sedels.  Such  oiodels  are  seed. -Markov  stochastic  processes  where 
a given  state  of  the  process  represents  a specified  level  of  deteri- 
oration of  the  system.  By  convention,  the  state  space  Is  a subset  of 
the  real  nuad>ers  with  Increasing  nundters  denoting  increasing  deteri- 
oration levels  and  an  Increasing  probability  of  total  failure  as  well 
as  Increasing  maintenance  costs. 

A semi -Markov  replacement  model  has  the  following  characteristics: 
1)  when  the  system  has  a Juiq)  In  Its  deterioration  level,  the  magnitude 

of  the  jump  depends  only  on  the  deterioration  level  Inssedlately  before 

» • . 

the  jump  and  2)  the  length  of  time  between  jumps  may  g^Wrna<tjl)^ 


arbitrary  distribution  function  dependent  on  the  current  level  of  “detAti^-u ' 
oration.  The  replacement  policies  studied  with  this  model  artf' thA 
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method  for  obtaining  either  the  optimal  Htnte  dependent  policy  or  the 
optimal  state-age  dependent  policy  Is  to  use  the  standard  semi-Markov 
decision  theory  approach  as  given  by  Howard  (1971).  The  application  of 
decision  theory  to  the  semi-Markov  replacement  problem  Is  given  In 
detail  by  Kao  (1973).  Feldman  (1976,  1977)  develops  an  alternate  method 
for  determining  the  optimal  state  dependent  replacement  policy  by  using 
Markov  renewal  theory  to  obtain  a closed  form  expression  for  the  cost. 
The  purpose  of  this  research  Is  to  extend  these  procedures  to  Include 
state-age  dependent  replacement  rules  and  determine  Its  computational 
efficiency. 

The  research  was  extremely  successful.  A closed  form  expression 
.w..  .e  long  term  average  replacement  cost  was  derived  for  state-age 
replacement  policy.  An  algorithm  using  the  closed  form  expression 
was  developed  and  compared  with  the  standard  policy  iteration-decision 
theory  approach.  Forty  five  sample  problems  were  compared  ranging 
from  a problem  of  four  states  to  a problem  of  thirty  states.  In  all 
cases  the  algorithm  developed  here  was  significantly  faster  than  the 
traditional  approach.  In  general,  the  advantage  of  using  the  closed 
form  algorithm  over  policy  Iteration  Increased  as  the  else  of  the  state 
space  Increased.  The  ratio  of  the  execution  time  for  the  new  algorithm 
to  the  execution  time  for  policy  Iteration  ranged  from  1.61  to  18.70. 

A proof  that  the  algorithm  will  always  converge  has  not  yet  been 
finished.  As  soon  as  the  proofs  are  completed,  a paper  giving  the 
algorithm  and  the  comparisons  will  be  submitted  to  Operations  Research. 


The  Master's  thesis  of  Tllden  N.  Mlkel  is  included  in  the  appendix 
which  gives  the  details  of  the  algoritha  and  the  coaputer  prograa 
used  for  making  conparisons. 
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ABSTRACT 


An  optimal  state-age  replacement  for  a semi -Harkov  deter- 
ioration process.  (December  1977) 

Tilden  Newton  Mikel,  B.S.,  Texas  A&M  University 
Chairman  of  Advisory  Committee:  Dr.  Richard  M.  Feldman 

A number  of  models  and  corresponding  replacement  policies  have 
been  developed  for  stochastically  deteriorating  systems  with  both 
military  and  industrial  applications.  One  such  class  of  problems 
can  be  modeled  as  a discrete  time  finite  state  semi-Markov  process 
with  the  deterioration  of  the  system  being  described  by  a Markov 
chain.  Replacement  policies  possible  for  such  a model  include  those 
based  only  on  the  state  of  the  system,  only  on  the  age  of  the  system 
and  on  a combination  of  the  state  of  the  system  and  its  sojourn 
time  in  that  state.  The  latter  of  these,  a state-age  replacement 
policy,  is  the  policy  of  concern  to  this  paper.  The  only  procedure 
to  find  an  optimal  state-age  replacement  policy  is  the  State-Age 
Dependent  Policy  developed  by  Kao  (1972),  requiring  solution  by 
Policy  Iteration  (Howard,  1960).  This  paper  derives  a closed  form 
expression  for  the  expected  long  term  average  cost  per  unit  time  and 
using  the  properties  of  the  Harkov  chain  and  additional  constraints, 
develops  a systematic  search  technique  for  the  Optimal  State-Age 
Replacement  Policy. 
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1.  INTRODUCTION 

A number  of  models  and  corresponding  replacement  policies  have 
been  developed  for  stochastically  deteriorating  systems  In  an  effort 
to  reduce  system  operation  and  maintenance  costs.  Applications  for 
such  replacement  models  and  policies  abound  In  both  Industry  and 
the  military.  Among  these  models  Is  a specific  class  of  problems 
that  are  referred  to  as  "Markovian  Replacement  Models"  (Kao,  1972). 
In  these  models  the  deterioration  of  the  system  Is  represented  by 
the  change  In  state  of  the  system  which  follows  the  transition- 
probability  matrix  of  a Markov  chain.  The  state  space  of  this 
Markov  chain  Is  {0,  1,  ....  L},  where  state  0 represents  a new  sys- 
tem and  state  L represents  a totally  failed  system.  Given  that  the 
system  starts  In  state  0 and  no  action  Is  taken  to  replace  It, 
deterioration  will  cause  transitions  to  successively  less  desirable 
states  until  eventually  state  L Is  reached  and  total  failure  occurs. 
If  the  costs  of  replacing  a failed  system  are  greater  than  replacing 
the  system  at  some  time  prior  to  failure,  then  a replacement  policy 
based  on  the  current  state  of  occupancy  may  result  In  a lower  over- 
all cost  to  the  user.  The  optimal  replacement  policy  for  such  a 
system  will  be  that  policy  that  best  balances  the  costs  associated 
with  repair,  replacement,  and  operation  over  some  defined  period  of 
time.  This  balance  may  be  easily  approached  by  assuming  that  the 

I system  will  be  needed  In  service  Indefinitely  and  that  the  objective 

. 

The  style  and  format  of  this  thesis  follow  that  of  the  Journal 
i of  Applied  Probability. 


of  the  problem  Is  minimization  of  the  long  term  average  cost  per 
unit  time.  This  basic  Markov  model  was  first  suggested  by  Derman 
(1962)  and  has  since  been  studied  by  Barlow,  Proschan,  and  Hunter 
(1965),  Ross  (1970),  and  Taylor  (1975). 

One  obvious  disadvantage  of  the  Markovian  Replacement  Model 
is  that  a Markov  process  does  not  consider  that  the  longer  the 
system  remains  in  a given  state,  the  more  likely  it  is  to  deteri- 
orate or  fail.  An  increasing  likelihood  of  failure  with  age  is 
very  common  in  practice.  Age  may  be  incorporated  into  the  model 
by  using  a semi -Markov  process  where  the  deterioration  among  states 
follows  a Markov  chain  and  the  time  spent  in  a state  is  a random 
variable  dependent  on  the  current  state  of  occupancy.  Cinlar  (1975) 
and  Ross  (1970)  describe  the  theory  behind  the  semi-Markov  process, 
and  optimal  replacement  rules  for  a finite  state,  discrete  time 
system  are  developed  by  Kao  (1972).  Kao  develops  three  types  of 
replacement  policies  based  on  information  that  is  available  to  the 
decision  maker:  (1)  a State  Dependent  Policy,  (2)  a State-Age 
Dependent  Policy,  and  (3)  an  Age  Dependent  Policy.  Kao  shows  that 
the  State-Age  Dependent  policy  is  in  general  superior  to  the  others, 
but  it  may  not  be  worth  the  additional  expense  to  keep  track  of 
the  state  of  the  system  and  the  length  of  time  it  has  been  in  that 
state.  Feldman  (1976)  considers  both  finite  and  infinite  state 
spaces,  and  develops  a control  limit  policy  for  replacement  using 
a closed  form  expression  for  the  expected  long  term  average  cost 
per  unit  time.  This  method  is  superior  to  the  Policy  Iteration 
Method  of  Howard  (1960)  and  has  been  shown  to  be  computationally 


more  efficient  than  the  method  of  Kao  (Lampe,  1977).  The  computa- 
tional efficiency  is  of  particular  significance  since  all  three 
methods  arrive  at  identical  control  limit  policies  with  the  same 
expected  long  term  average  cost  per  unit  time. 

The  purpose  of  this  paper  is  to  develop  a closed  form  expres- 
sion for  the  expected  long  term  average  cost  per  unit  time  for  a 
state-age  replacement  policy  and  to  develop  an  algorithm  using 
this  expression  to  find  the  optimal  state-age  replacement  policy. 
This  method  will  then  be  compared  to  Kao's  method  using  policy 
iteration  to  verify  the  optimality  of  the  new  method  and  to  demon- 
strate its  computational  efficiency.  In  Section  2,  the  parameters, 
conditions,  and  notation  of  the  problem  are  defined;  in  Section  3, 
a summary  of  Kao's  method  for  the  State-Age  Dependent  Policy  using 
Policy  Iteration  is  presented;  in  Section  4,  the  expected  long 
term  average  cost  per  unit  time  expression  and  a selective  search 
algorithm  are  developed  to  find  the  optimal  state-age  replacement 
policy;  in  Section  5,  an  example  problem  is  solved  by  each  method; 
and  Section  6 presents  a comparison  between  the  methods  for  a 
variety  of  problems  using  FORTRAN  programming  for  each  method. 
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2.  PROBLEM  STATEMENT 

Consider  a system  whose  underlying  condition  can  be  identified 
at  any  distinct  point  in  time  by  classifying  it  in  one  of  a finite 
number  of  states  0,  1,  ....  L.  This  set  of  states  will  be  denoted 
by  E.  The  system  is  subject  to  a sequence  of  randomly  occuring 
deterioration  forces,  each  of  which  causes  some  random  amount  of 
damage  to  the  system.  These  forces  may  be  shocks  to  the  system, 
the  effects  of  normal  wear  and  tear,  abnormal  stress  on  the  system, 
fatigue,  power  fluctuations  or  surges,  and  numerous  other  factors 
that  could  cause  deterioration  in  a given  system.  This  narrative 
will  follow  the  format  of  Feldman  (1976)  and  refer  to  all  such 
deterioration  forces  as  shocks.  A new  system  prior  to  any  deteri- 
oration is  classified  as  being  in  state  0;  a completely  failed 
system  is  in  state  L.  The  intermediate  states,  state  1 through 
state  L-1,  represent  discrete,  ordered  degrees  of  deterioration 
short  of  total  failure.  Since  each  shock  is  of  random  magnitude 
and  damage  to  the  system  is  the  cumulative  damage  caused  by  the 
shocks,  the  system  can  only  deteriorate  until  it  is  replaced  or 
a total  failure  occurs.  Replacement  upon  failure  is  assumed  to  be 
mandatory,  as  it  is  of  no  economic  advantage  to  delay  replacement 
if  the  system  is  nonproductive.  This  deterioration  is  assumed  to 
follow  a semi -Markov  orocess.  The  shocks  to  the  system  will  be 
assumed  to  occur  at  the  end  of  the  discrete  time  intervals  according 


j ^ to  a discrete  holding  time  nwss  function. 
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It  Is  necessary  to  define  the  notation  to  be  used  and  to 
establish  some  additional  conditions  on  the  problem.  The  trans- 
ition matrix  for  the  Imbedded  Markov  chain  of  the  semi -Markovian 
deterioration  process  Is  an  (L+1)  x (L+1)  matrix  consisting  of 
Individual  elements  p(1,j)  for  1 , j < E,  where  p(1,j)  Is  the  pro- 
bability of  moving  from  state  1 to  state  j assuming  there  Is  no 
replacement  before  failure.  Since  the  system  can  only  deteriorate, 
then 

p(1.j)  = 0 for  j < I and  1 L 

and 

L 

T p(1»j)  = 1 for  all  1 « E. 

j = 0 

The  policy  vector  k Is  the  set  of  decisions  at  each  state  (kq, 

<1,  ....  iC|^),  where  Indicates  the  decision  to  replace  the  system 
given  that  It  has  been  In  state  1 for  units  of  time.  Each 
has  a lower  bound  of  zero.  Indicating  that  the  decision  Is  to 
replace  the  system  Immediately  upon  reaching  state  1.  The  upper 
bound  of  each  tc^  Is  the  maximum  sojourn  time  allowed  In  state  1 
by  the  holding  time  mass  function.  Indicating  that  the  decision  Is 
to  not  replace  the  system  as  long  as  It  Is  In  state  1.  The  decision 
at  state  L Is  limited  to  K|^  = 0 to  force  Immediate  replacement  upon 
failure.  The  semi-Markov  kernel  Q(1,j,t)  Is  defined  as  the  pro- 
bability of  moving  from  state  1 to  state  J In  the  time  interval  [0,t]. 
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The  cumulative  probability  distribution  for  the  sojourn  time  in 
state  i is  denoted  by  H(i,*)  for  i « E and  is  given  by 

H(i,t)  = Z Q(i,j,t)  for  i,j  c E,  t>  0. 
j « E 

A constraint  imposed  on  the  system  is  that  the  cumulative  proba- 
bility distribution  of  sojourn  times  is  nondecreasing  in  i.  This 
property  means  that  the  sojourn  times  stochastically  decrease  as 
i increases  and  it  implies  that  the  mean  sojourn  times  are  decreas- 
ing in  i,  a condition  Kao  imposes  in  his  model. 

The  optimal  replacement  policy  is  defined  as  the  policy  giving 
the  minimum  long  term  average  cost  per  unit  time.  There  is  an 
occupancy  cost  rate  a^  associated  with  each  unit  of  time  spent  in 
state  i;  this  cost  represents  the  costs  of  operation  and  routine 
maintenance.  The  replacement  cost  c^  is  the  total  cost  of  replacing 
the  system  in  state  i.  It  may  consist  of  a base  cost  for  replace- 
ment b^  and  an  additional  cost  rate  per  unit  time  d^  while  the 
system  is  out  of  service  for  replacement.  The  mean  time  for  the 
accomplishment  of  a replacement  in  state  i is  defined  as  7^.  The 
total  replacement  cost  c^  and  the  occupancy  cost  rate  a^  are  both 
assumed  to  be  nondecreasing  in  i in  order  to  insure  a reasonable 
form  for  the  optimal  solution. 
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3.  KAO'S  METHOD 

Kao's  method  for  the  State-Age  Dependent  Policy  utilizes  a 
search  technique  called  Policy  Iteration  which  was  developed  by 
Howard  (1960).  This  procedure  Involves  selection  of  an  Initial 
policy  jc  = (kq.  I'll  ...t  and  through  successive  evaluations 
and  Improvements  the  policy  Is  made  to  converge  to  the  optimal. 

Initially  given  are  the  equations  for  1 = 0,  1 L: 

L 

v(1,k)  + g(K)T(i,K)  = f(1,K)  + I q(1,j,tc)  v(j) 

" “ ~ “ j“0  - 

where  t(1,£)  Is  the  mean  waiting  time  under  policy  k for  state  1, 

) f(1f<)  Is  the  expected  cost  of  state  1,  and  q(1,j,^)  Is  an  element 

of  the  semi-Markov  kernel  under  policy^.  These  equations  are 

solved  by  letting  v(L)  = 0 and  the  relative  cost  rate  g(£)  is 

obtained  for  the  policy.  Thus  v(1)  Is  the  relative  cost  of  state 

1 when  v(L)  = 0 and  g(iO  Is  the  relative  cost  rate  of  the  policy 

1^.  The  relative  values  of  v(1)  are  used  to  find  the  decision 

that  will  minimize  the  test  quantity  r(1,  k^)  given  by 

L 

f(1.0  + E qd.j.x)  v(j,K)  - v(1,^ 
r(1,  = jfO . 

t(1,k) 

When  a new  has  been  determined  for  each  1,  a new  policy  £ 

* . has  been  defined.  The  new  policy  Is  then  evaluated  by  solving 

for  the  cost  rate  and  relative  costs  of  the  states.  Iterations 
continue  In  an  attempt  to  Improve  the  policy  until  there  Is  no 

I 

i change  In  the  policy  from  one  Iteration  to  the  next.  This  repll- 


i 


cation  of  Identical  pol Icies  Indicates  that  the  optimal  replacement 
policy  has  been  reached.  The  cost  rate  g(0  Is  the  long  term 
expected  average  cost  per  unit  time  for  the  optimal  State-Age 
Dependent  Policy. 

Although  this  procedure  will  generally  find  the  optimal  state 
age  replacement  policy,  the  solutions  to  L + 1 simultaneous 
equations  for  each  Iteration  are  quite  time  consuming.  As  the 
problem  size  Increases,  the  computations  also  Increase,  as  does 
the  size  of  required  array  space  for  a computer  software  package 
using  Kao's  method.  If  multiple  solutions  exist,  the  method  can 
take  an  Inordinate  amount  of  time  to  recognize  that  the  solution 
Is  optimal.  Round-off  error  can  also  compound  In  a very  large 
problem  as  matrix  Inversion  Is  repeated  through  several  Iterations 

A program  listing  for  Kao's  method  coded  In  FORTRAN  IV  Is 
found  In  Appendix  B.  An  example  problem  Is  solved  In  Section  5. 
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4.  THE  OPTIMAL  STATE-AGE  REPLACEMENT  POLICY  ALGORITHM 

Prior  to  the  development  of  an  algorithm  to  find  an  optimal 
state-age  replacement  policy,  an  equation  for  the  expect  long 
term  average  cost  per  unit  time,  4»(£),  must  be  derived.  This  cost 
Is  a function  of  the  expected  mean  sojourn  time  In  each  state,  the 
costs  of  maintenance  and  replacement  In  each  state,  and  the  associ- 
ated probabilities  of  deterioration  and  replacement  among  the 
states. 

Let  { Z^,  t - 0}  be  a semi-Markov  process  with  state  space 
E as  previously  defined,  where  L denostes  the  failed  state.  Now 
let  (X^,  n = 0,  1,  ....  LI  be  the  Imbedded  Markov  chain  associated 
with  the  semi-Markov  process  Z using  decision  i^.  Let  P denote  its 
transition  matrix  (P  actually  depends  on  k but  for  notational  con- 
venience this  dependence  will  not  be  shown).  Then  P Is  defined  by 

P(1.j)  * q(1.j.£)  for  1,j  « E under  the  policy 

Let  A be  a state  not  In  E and  let  P(1,a)  = 1 - z P(1,j). 

j‘E 

Thus  A represents  the  "death"  of  the  Markov  chain  X. 

Define  the  random  variable  c by 

C = 1nf{n:X_«  L)A  sup  { n:X„«  E} . 
n n 

Now  the  expected  cost  of  replacement  may  be  defined  as 

E.[c(X  )]  = c(L)P(1,L)  + c(1)P(1,A)  + z P(1,k)E.[c(X  1] 
' ^ k€E\{L> 


I 
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f 

i 


I 


! 


* 2 R(i.k)[c(L)P(k,L)  + c(k)P(k.A)]. 

k€  EUL) 


Since 


then 


R(i.j)  = 2 P (i.j),  and  P(L,L)  = 0 

n = 0 


R P(i.j)  = R(i.j)  - Ki.j) 


and  thus 


E.[c(X  )]  = E R(i,j)  c(k)  P(k,A). 

• S I-  - r 


k*  E 

It  should  be  noted  that 


P(i,  A ) = 1 - P(i.E), 
P(L,j)  = 0 for  j E, 
and  P(L,  A ) = 1. 


It  is  now  necessary  to  find  the  expected  cycle  time  E^[c].  The 
expected  sojourn  time  in  state  i can  be  written  as 
m(i)  = [1  - Q(i.E,t)]  dt. 

Recalling  that  the  mean  replacement  time  at  state  i is  it 
follows  that 

E.[c]  = m(i)  +7  P(i,L)  + 7.  P(i,A)  + E P(i,k)  E.[t] 

’ ktEUL) 

* r R(i,k)[m(k)  + 7 P(k,L)  + 7.  P(k.A)] 
k«E\{L)  * 

■ R(i,L)  T.  E R(i,k)  m(k)  7.  P(k,A) 
k«E\{L} 


n 


A 

and  E.Ct]  = z R(i,k)  [in(k)  + t.  P(k,A)]  where  m(L)  = 0. 

^ k€  E 

The  Markov  renewal  kernel  R(i,j)  may  now  be  shown  to  be  R(i,j)  = 
[I  - P]”^  for  a given  policy  k.  The  inverse  of  this  upper  right 
triangular  matrix  may  be  easily  computed  as  follows: 
r(i,0)  = 0 
r(i,i)  = 1 

r(i,i+l)  = r(i,i)  q(i,i+l  ,ic^.) 

r(i,i+2)  = r(i,i)  q(i,i+2,K^)  + r(i,i+l)  q(i+l ,i+2,K.^^ ) 


Since  the  system  will  always  start  a cycle  in  state  0 and  return 
to  state  0 upon  replacement,  the  only  elements  of  the  Markov 
renewal  matrix  of  concern  to  this  problem  are  r(0,j)  for  all  j « E. 

The  expected  long  term  average  cost  per  unit  time  can  now 
be  defined  as  the  total  expected  replacement  and  maintenance  costs 
per  cycle  divided  by  the  expected  cycle  time,  or 

L 

z r(0,i)  [m(i)  a.  + c.  P(i,0)] 
i = 0 ' ^ 

• 

L 

z r(0,i)  [m(i)  + t.  P(i,0)] 
i « 0 ^ 

where  ■ b^  ♦ and  P{i,0)  ■ P(i,A).  This  expression  is  valid 
for  any  feasible  policy  k;  all  that  now  remains  is  the  development 
of  an  efficient  algorithm  to  find  the  optimal  state-age  replacement 
policy. 
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An  obvious  initial  policy  for  beginning  a search  for  the  optimal 
is  to  replace  the  system  only  when  it  fails.  This  may  be  accomp- 
lished by  setting  k.  equal  to  its  maximum  value  for  i = 0,  1,  ....  L-1 
and  = 0.  Evaluation  of  this  policy  gives  the  expected  long  term 
average  cost  per  unit  time  for  the  basic  problem  without  early  re- 
placement. Starting  in  state  L-1,  the  decision  at  each  state  may 
be  varied  to  find  the  minimum  cost,  all  other  decisions  remaining 
constant.  By  passing  through  the  states  from  L-1  to  0 and  adopting 
the  minimum  cost  decision  as  part  of  the  new  policy  as  the  search 
passes  through  state  i,  the  resulting  policy  will  be  at  least  as 
good  as  the  old  policy,  and  it  will  be  better  if  the  old  policy  was 
not  the  optimal.  It  is  not  necessary  to  evaluate  every  possible 
at  each  state  i.  Since  the  costs  are  non-decreasing  in  i and 
the  cumulative  probability  distribution  of  sojourn  times  is  also 
non-decreasing  in  i,  This  is  a logical  conclusion  in 

that  it  could  not  be  of  economic  advantage  to  remain  in  a more  ex- 
pensive state  longer  than  the  limiting  time  of  a less  expensive  state. 
Therefore,  in  varying  the  decision  at  state  i,  evaluations  are  only 
made  between  and  the  upper  bound  of  as  determined  by  the 

holding  time  mass  function  for  the  first  iteration. 

A single  pass  may  not  yield  the  optimal  solution;  since  all 
states  are  initially  set  at  the  maximum  decision  value,  a bias  may 
exist  in  the  decisions  computed  for  the  higher  numbered  states. 

This  bias  may  be  removed  by  successive  passes  through  the  states, 
noting  that  Kj  may  only  decrease  for  subsequent  passes  through 


) 
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state  j,  j * E.  Thus  the  search  space  is  decreased  somewhat  with 

each  successive  iteration,  as  the  maximum  k.  is  the  old  policy 

3 3 

and  the  minimum  is  the  current  policy  . This  non-increasing 
property  of  decisions  in  successive  iterations  is  again  due  to  the 
non-decreasing  properties  of  the  system  parameters.  The  only  cause 
for  a change  in  kj  is  a subsequent  new  minimum  found  at  where 
i < j.  Since  the  new  will  be  smaller  than  the  old  k^.  its  only 
possible  influence  on  k-  will  be  to  make  it  smaller  than  the  pre- 

si 

vious  decision.  If  the  state  by  state  iterative  passes  are  continued 
until  one  complete  pass  is  made  with  no  change  in  the  overall 
policy,  the  optimal  state-age  replacement  policy  has  been  found. 

It  must  be  noted  at  this  point  that  two  assumptions  have  been 
made  and  used  in  development  of  the  algorithm  with  no  proof  as  to 
their  validity.  The  first  is  that  the  decision  will  be  non- 
increasing as  i increases.  The  other  is  that  the  decision  k-  will 
be  non- increasing  through  successive  iterations.  Both  assumptions 
are  based  on  the  author's  intuition  given  the  increasing  nature  of 
the  costs  as  the  state  increases  and  the  increasing  likelihood  of 
failure  as  the  state  and  sojourn  times  increase.  They  have  been 
shown  to  be  true  in  the  solution  of  the  sample  problems  to  be  ref- 
erenced in  Sections  5 and  6.  A formal  proof  of  these  properties  is 
beyond  the  scope  of  this  thesis. 

A listing  of  the  Optimal  State-Age  Replacenent  Policy  Algorithm 
coded  in  FORTRAN  IV  may  be  found  in  Appendix  C.  An  example  problem 
is  solved  in  Section  5 using  the  algorithm. 
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5.  AN  EXAMPLE  PROBLEM 

To  Illustrate  the  solution  procedures  for  Kao's  State-Age 
Dependent  Policy  and  the  Optimal  State-Age  Replacement  Policy 
presented  In  Section  4,  an  example  Is  given  with  the  following 
parameters: 


The 

state 

space 

E 

Is 

(0,  1. 

2. 

3). 

The 

replacCTont 

costs 

are: 

"o  = 

2.0 

( 

*>0 

= 1.0, 

*10 

= 1.0, 

^0  ' 

1.0 

2.0 

( 

= 1.0, 

^1 

* 1.0, 

"1  “ 

1.0 

2.0 

( 

‘’2 

= 1.0, 

‘*2 

» 1.0, 

^2  " 

1.0 

"3  = 

6.0 

( 

*>3 

= 2.0, 

‘*3 

= 2.0, 

■^3  * 

2.0 

The  transition  probability  matrix  P Is 


0.0 

0.6 

0.2 

0.2 

0.0 

0.0 

0.7 

0.3 

0.0 

0.0 

0.0 

1.0 

0.0 

0.0 

0.0 

1.0 

The  state  holding  time  probabilities,  h(1,t),  are: 

h(O.O)  - 0.0,  h(0,l)  = 0.1,  h(0,2)  * 0.18,  h(0,3)  - 0.216, 

h(0.4)  - 0.454,  h(0,5)  - 0.050,  H(0,5)  = 1.0, 

h(l,0)  - 0.0,  h(l,l)  * 0.2,  h(l,2)-0.14,  h(l ,3)  » 0.528, 

h(l,4)  - 0.119,  h(l,5)  - 0.013,  H(l,5)-1.0, 

h(2,0)  - 0.0,  h(2.1)  - 0.3,  h(2,2)  - 0.28,  h(2,3)  - 0.378, 

h(2,4)  - 0.040,  h(2,5)  * 0.002,  H(2,5)  - 1.0, 

h(3,0)  * 0.0,  with  h(3,j)  undefined  for  j^O. 
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Kao's  Method 

Prior  to  beginning  the  Policy  Iteration  step  of  Kao's  method, 
the  modified  transition  probabilities,  mean  holding  times,  mean 
waiting  times,  and  expected  cost  rates  must  be  calculated.  These 
are  tabulated  below  In  Table  1. 


Table  1.  Data  used  for  State-Age  Dependent  Policy  (Kao's  Method) 


^ •'1  PlO  Pll  Pi2  Pl3  ^10  ^1j  f(i,K^.) 


» 

0 

0 

1.000  0.000  0.000  0.000 

1.0 

0.0000 

1.000 

2.0000 

1 

0.900  0.060  0.020  0.020 

2.0 

1 .0000 

1.900 

1 .4737 

2 

0.720  0.168  0.056  0.056 

3.0 

0.6429 

2.620 

0.2748 

3 

0.504  0.298  0.099  0.099 

4.0 

2.2339 

3.124 

1.1613 

4 

0.050  0.570  0.190  0.190 

5.0 

3.0779 

3.174 

1.0158 

- 

5 

0.000  0.600  0.200  0.200 

6.0 

3.1740 

3.174 

1 .0000 

1 

0 

1.000  0.000  0.000  0.000 

1.0 

0.0000 

1.000 

2.0000 

1 

0.800  0.000  0.140  0.060 

2.0 

1.0000 

1.800 

1 .4444 

2 

0.660  0.000  0.238  0.102 

3.0 

1.4118 

2.460 

1.2683 

3 

0.132  0.000  0.608  0.260 

4.0 

2.3779 

2.592 

1 .0509 

4 

0.013  0.000  0.691  0.296 

5.0 

2.5735 

2.605 

1 .0050 

5 

0.000  0.000  0.700  0.300 

6.0 

2.6050 

2.605 

1 .0000 

! 

j 

2 

0 

1.000  0.000  0.000  0.000 

1.0 

0.0000 

1.000 

2.0000 

i 

1 

0.700  0.000  0.000  0.300 

2.0 

1 .0000 

1.700 

1.4118 

1 

1 

i 

2 

0.420  0.000  0.000  0.580 

3.0 

1 .4828 

2.120 

1.1981 

f 

[ 

3 

0.042  0.000  0.000  0.958 

4.0 

2.0814 

2.162 

1.0194 

j 

4 

0.002  0.000  0.000  0.998 

5.0 

2.1583 

2.164 

1.0009 

i 

5 

0.000  0.000  0.000  1.000 

6.0 

2.1640 

2.164 

1.0000 

1 

3 

0 

1.000  0.000  0.000  0.000 

2.0 

0.0000 

2.000 

3.0000 

Selecting  the  decisions  with  the  minimum  expected  cost  rates,  the 
Initial  policy  £ Is  (5, 5, 5,0).  The  gain  equations  will  be: 
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v(0,£)  + 3.174  g(K)  = 3.174  + 0.0  v(0,£)  + 0.6  v(1,ic) 

+ 0.2  v(2,k)  + 0.2  v(3.k) 

v(1,k)  + 2.605  g(K)  = 2.605  + 0.0  v(0,£)  + 0.0  v(1,£) 

+ 0.7  v(2,ic)  + 0.3  v(3,jc) 

v(2,£)  + 2.164  g(£)  = 2.164  + 0.0  v(0,£)  + 0.0  v(l,£) 

+ 0.0  v{2,£)  + 1.0  v(3,£) 

v(3,£)  + 2.000  g(£)  = 6.000  + 1.0  v(0,£)  + 0.0  v(l,£) 

+ 0.0  v(2,£)  + 0.0  v(3,£) 

Setting  v(3,£)  = 0.0,  the  gain  equations  niay  be  solved  to  yield 

gU)  = 1.49513 

v(0,O  = -3.00974 

v(l,£)  = -2.03984 

v(2,£)  = -1.07146 

v(3,£)  = 0.0 

Solution  of  the  r equations  yields  a new  policy  of  £ = (3, 0,0,0). 

Four  iterations  are  required  to  find  the  optimal  policy,  £*  = (4, 2, 0,0) 
with  expected  long  term  average  cost  rate  of  i|)(k*)  = 0.32315.  A 
tabulation  of  the  iterations  may  be  found  in  Table  2.  Execution  of 
the  FORTRAN  program  for  this  method  required  0.0671  seconds  under  a 
FORTRAN  IV-H  compiler. 

The  Optimal  State-Age  Algorithm 

The  same  problem  can  be  solved  using  the  Optimal  State-Age  Replace- 
ment Algorithm  in  only  two  iterations.  The  initial  policy  is  the  same 
as  that  of  Kao's  method,  jc  = (5, 5, 5,0).  Step-wise  calculations  are 
provided  in  Table  3.  The  solution  is  identical  to  Kao's  method,  with 
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K* 


(4, 2, 0,0)  and  <((k*)  = 1.32315.  Execution  of  the  FORTRAN  program 


took  only  0.0232  seconds,  a 2.9  to  1 advantage  over  Kao's  method. 


Table  2.  Results  from  State-Age  Dependent  Policy  (Kao's  Method) 


State 

mr 

Iteration 

Iteration 

Iteration 

Iteration 

1 

2 

3 

4 

i 

K . 

1 

^i 

'i 

0 

0 

2.0000 

2.0000 

2.0000 

2.0000 

1 

1.5564 

1.5358 

1 .5276 

1.5277 

2 

1 .4428 

1 .4011 

1 .3843 

1.3846 

3 

1.4109* 

1.3489* 

1.3241 

1 .3243 

4 

1 .4861 

1.3693 

1 .3225* 

1.3232* 

5 

1 .4951 

1 .3722 

1.3229 

1.3236 

1 

0 

1 .0301* 

1 .3489 

1 .6344 

1.6318 

1 

1.1567 

1.2434 

1 .4057 

1 .4043 

2 

1.1863 

1 .2035* 

1 .3421* 

1 .3232* 

3 

1 .4335 

1.2841 

1 .4051 

1 .4045 

4 

1.4888 

1.3031 

1.4249 

1 .4244 

5 

1 .4951 

1.3053 

1.4273 

1 .4268 

2 

0 

0.0618* 

1 .3489* 

1 .3241* 

1 .3232* 

1 

0.8027 

1.6115 

1 .6057 

1.6054 

2 

1.1073 

1 .7944 

1.7963 

1.7964 

3 

1 .4566 

2.1815 

2.1920 

2.1924 

4 

1.4933 

2.2229 

2.2344 

2.2348 

5 

1 .4951 

2.2251 

2.2366 

2.2370 

3 

0 

1 .4951* 

1 .3489* 

1 .3241* 

1.3232* 

Pol  icy 

Evaluation 

gU) 

1.49513 

1.34889 

1.32406 

1 .32315 

v(0,O 

-3.00974 

-3.30220 

-3.35187 

-3.35368 

v(1.k) 

-2.03984 

-2.65109 

-2.98630 

-2.98547 

CM 

> 

-1.07146 

-2.65109 

-2.67593 

-2.67683 

v(3,ic.) 

0.0 

0.0 

0.0 

0.0 

1 

! 

i 

f 

i 


» (4, 2,0,0) 
<,(£*)  - 1.32315 
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Table  3.  Results  of  Optimal  State-Age  Replacement  Policy  Algorithm 


Iter. 

# 

State 

i 

Decision 

•^i 

r(d.O)  ■ 

Renewal 

r(O.l) 

Kernel 

rCff.?) 

>■(0.3) 

Cost  Rate 

i|)(£) 

1 

2 

0 

1 .0000 

0.6000 

0.6200 

0.3800 

1.34984* 

1 

1 .0000 

0.6000 

0.6200 

0.5660 

1.38972 

2 

1 .0000 

0.6000 

0.6200 

0.7396 

1 .42743 

3 

1 .0000 

0.6000 

0.6200 

0.9740 

1 .48869 

4 

1 .0000 

0.6000 

0.6200 

0.9988 

1 .49482 

5 

1 .0000 

0.6000 

0.6200 

1 .0000 

1 .49513 

1 

0 

1 .0000 

0.6000 

0.2000 

0.2000 

1.36580 

1 

1.0000 

0.6000 

0.2840 

0.2360 

1 .34092 

2 

1.0000 

0.6000 

0.3428 

0.2612 

1 .32340* 

3 

1 .0000 

0.6000 

0.5646 

0.3562 

1.34443 

4 

1 .0000 

0.6000 

0.6145 

0.3777 

1.34928 

5 

1 .0000 

0.6000 

0.6200 

0.3800 

1.34984 

0 

2 

1 .0000 

0.1680 

0.0960 

0.0731 

1.37228 

3 

1 .0000 

0.2976 

0.1700 

0.1296 

1.32406 

4 

1.0000 

0.5700 

0.3257 

0.2481 

1.32315* 

5 

1.0000 

0.6000 

0.3428 

0.2612 

1.32340 

2 

1 

0 

1 .0000 

0.5700 

0.1900 

0.1900 

1.36393 

1 

1.0000 

0.5700 

0.2699 

0.2242 

1.34008 

2 

1.0000 

0.5700 

0.3257 

0.2481 

1.32315* 

0 

2 

1.0000 

0.1680 

0.0960 

0.0731 

1.37228 

3 

1 .0000 

0.2976 

0.1700 

0.1296 

1.32406 

4 

1.0000 

0.5700 

0.3257 

0.2481 

1.32315** 

K * (4.2, 0.0) 

= 1.32315 
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6.  A COMPARISON  OF  THE  METHODS 

In  order  to  compare  the  Optimal  State-Age  Algorithm  with  Kao's 
method,  both  solution  procedures  have  been  programmed  in  FORTRAN  IV. 

A main  program  was  developed  to  do  all  data  input  operations,  prelim- 
inary calculations  required  by  both  methods,  and  checks  to  insure  that 
the  probiMi  parameters  are  acceptable.  The  main  program  then  calls 
a subroutine  for  Kao's  method,  recording  the  execution  time  of  the 
subroutine;  it  then  calls  a subroutine  for  the  Optimal  State-Age 
Algorithm  and  again  records  the  execution  time.  Output  for  the  pro- 
blem solution  is  provided  by  the  individual  subroutines.  The  exe- 
cution times  are  printed  by  the  main  program  prior  to  termination. 

Thus  an  unbiased  comparison  between  the  methods  is  provided  based  only 
on  the  separate  solution  procedures  unhampered  by  the  calculations 
that  are  common  to  both  methods. 

The  results  of  forty-five  sample  problems  compared  by  this  com- 
puter program  are  contained  in  Table  4.  Some  of  the  problems  were 
read  in  on  data  cards  while  others  were  generated  by  a subroutine 
employing  a random  number  generator.  This  exercise  has  shown  the 
Optimal  State-Age  Algorithm  to  be  superior  to  Kao's  method.  A ratio 
of  execution  times  was  computed  as  a measure  of  the  computational 
efficiency,  with  Kao's  method  execution  time  in  the  denominator  and 
the  Optimal  State-Age  Algorithm  in  the  numerator.  The  smallest 
ratio  was  1.61  for  a four  state,  six  discrete  time  unit  problem. 

The  largest  ratio  was  18.70  for  a thirty  state,  twenty  discrete  time 
unit  problem.  A statistical  analysis  was  not  performed  on  these 
results,  as  inspection  of  the  ratios  clearly  implied  that  the  ratio 
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|r  is  a function  of  the  simplicity  of  the  solution  as  well  as  the  problem 

size.  This  is  caused  by  the  increase  in  the  number  of  iterations  that 
are  required  by  Kao's  method  for  a difficult  problem.  The  Optimal  State- 
Age  Algorithm  required  a maximum  of  three  iterations  for  the  largest 
ratio  problem,  while  Kao's  method  required  eleven  iterations.  In  general 
it  appears  that  the  new  method  will  be  in  the  neighborhood  of  two  or  more 
times  as  efficient  as  Kao's  method.  The  Optimal  State-Age  subroutine 
also  compiles  faster  than  Kao's  method's  subroutine. 

An  additional  benefit  of  the  Optimal  State-Age  Algorithm  is  a reduc- 
tion in  the  required  array  space  required  by  the  program.  This  array 
space  and  the  actual  ceding  will  vary  with  the  size  of  the  problem  under 
I consideration,  but  it  will  be  substantially  less  with  the  Optimal  State- 

I Age  Algorithm.  For  a thirty  state,  twenty  discrete  time  unit  problem, 

core  requirements  for  niain  program  and  Optimal  State-Age  subroutine  ex- 
cluding compiler  and  input  software  are  9606  bytes.  The  same  problem 
requires  169,372  bytes  of  core  for  Kao's  method.  Although  this  space 
savings  may  not  be  of  the  same  magnitude  in  smaller  problems,  it  does 
^ allow  the  Optimal  State-Age  Algorithm  to  be  employed  on  computers  with 

much  smaller  storage  areas  than  would  be  possible  with  Kao's  method. 

It  can  now  be  concluded  that  a more  efficient  method  has  been 
found  in  the  Optimal  State-Age  Algorithm.  Thus  the  objective  of  this 
research  has  been  achieved.  Although  the  method  is  restricted  to  a • 
special  class  of  problems  as  stated  in  Section  2,  the  concept  of  a 
^ systematic  search  using  a closed  form  cost  expression  has  been  shown 

* to  be  both  feasible  and  efficient.  Further  research  in  this  area  to 

develop  an  algorithm  for  a more  general  model  is  recotimended. 
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Table  4. 

Comparison  of  Execution  Times 

Number  of 

Range  of 

Execution  Time  in  Seconds 

Ratio 

States 

i 

A 

B 

A/B 

Kao's 

Optimal 

Method 

State-Age 

A1 gori thm 

4 

6 

0.01 22 

0.0076 

1.61 

4 

6 

0.0161 

0.0089 

1.81 

4 

6 

0.0191 

0.0091 

2.10 

4 

10 

0.01 51 

0.0082 

1.84 

4 

10 

0.0215 

0.0110 

1.95 

4 

10 

0.0201 

0.0110 

1.83 

4 

20 

0.0276 

0.0124 

2.23 

4 

20 

0.0327 

0.0142 

2.30 

4 

20 

0.0302 

0.0140 

2.16 

8 

6 

0.0462 

0.0145 

3.19 

8 

6 

0.0460 

0.0204 

2.25 

8 

6 

0.0489 

0.0187 

2.61 

8 

10 

0.0606 

0.01 76 

3.44 

8 

10 

0.0601 

0.0274 

2.19 

8 

10 

0.0623 

0.0255 

2.44 

8 

20 

0.1327 

0.0269 

4.93 

8 

20 

0.1182 

0.0461 

2.56 

8 

20 

0.1212 

0.0472 

2.57 

10 

6 

0.0884 

0.0187 

4.73 

10 

6 

0.0892 

0.0301 

2.96 

10 

6 

0.0748 

0.0296 

2.53 

10 

10 

0.1127 

0.0248 

4.54 

10 

10 

0.0966 

0.0421 

2.29 

10 

10 

0.0971 

0.0414 

2.35 

10 

20 

0.2118 

0.0398 

5.32 

10 

20 

0.1908 

0.0734 

2.60 

10 

20 

0.1091 

0.0477 

2.28 

20 

6 

0.4895 

0.0665 

7.36 

20 

6 

0.5992 

0.1321 

4.54 

20 

6 

0.3953 

0.1487 

2.66 

20 

10 

0.6999 

0.0922 

7.59 

20 

10 

0.5919 

0.1940 

3.05 

20 

10 

0.5954 

0.2171 

2.74 

20 

20 

1.1532 

0.1612 

7.15 

20 

20 

0.9020 

0.3624 

2.49 

20 

20 

0.9111 

0.1183 

7.70 

30 

6 

1 .4567 

0.1704 

8.55 

30 

6 

1.7972 

0.3578 

5.02 

30 

6 

1 .4970 

0.4750 

3.15 

30 

10 

1 .6848 

0.3051 

5.52 

30 

10 

1.7010 

0.5466 

3.11 

30 

10 

1.7292 

0.7362 

2.35 

30 

20 

5.0424 

0.4335 

11.63 

30 

20 

3.5923 

0.4257 

8.44 

30 

20 

3.5961 

0.1923 

18.70 
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APPENDIX  A 
MAIN  PROGRAM 

Following  is  a program  listing  for  the  main  program  used  to  eval- 
uate the  efficiency  of  Kao's  State-Age  Dependent  Policy  and  the  Optimal 
State-Age  Replacement  Policy  Algorithm  developed  in  this  paper.  The 
program  is  written  in  FORTRAN  IV  for  a H extended  compiler  and  should 
easily  adapt  to  any  machine  with  a FORTRAN  IV  compiler.  The  Subroutine 
TIMER  is  a system  subroutine  unique  to  the  Texas  AiM  University  Data 
Processing  System;  its  function  is  to  keep  track  of  actual  execution 
time  for  each  of  the  method  subroutines  and  it  can  be  easily  replaced 
by  a routine  to  access  the  computer's  internal  clock  prior  to  calling 
and  upon  return  from  the  method  subroutines.  This  listing  is  dimen- 
sioned for  a thirty  state,  twenty  possible  discrete  time  problem.  For 
larger  problems  or  to  save  array  space  in  smaller  problems,  the  user 
need  only  change  the  DIMENSION  and  COMMON  statements  to  reflect  the 
problem  size  desired. 
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APPENDIX  B 
SUBROUTINE  HPIM 

Following  is  a program  listing  of  Subroutine  HPIM,  a routine  to 
solve  for  the  optimal  state-age  replacement  policy  for  a semi-Markov 
deterioration  process.  The  solution  procedure  is  that  of  Kao  (1972) 
using  Howard's  Policy  Iteration  Method.  Solution  of  the  L+1  linear 
equations  is  done  by  inversion  in  place.  This  listing  is  for  a 
maximum  size  problem  of  thirty  states  and  twenty  possible  discrete 
holding  times  per  state.  For  larger  problems  or  to  save  array  space 
in  smaller  problems,  the  DIMENSION  and  COMMON  statements  may  be  mod- 
ified without  effecting  execution  of  the  subroutine.  All  output  is 
internally  coded  in  the  subroutine  so  that  it  may  be  used  without 
the  main  program  found  in  Appendix  A if  the  user  supplies  a program 
to  furnish  input  variables  and  call  the  subroutine. 
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APPENDIX  C 
SUBROUTINE  MFM 

t 

Following  is  a program  listing  of  Subroutine  MFM,  a routine  to 
solve  for  the  optimal  state-age  replacement  policy  for  a semi-Markov 
deterioration  process.  The  solution  procedure  is  the  one  developed  in 
this  paper,  the  Optimal  State-Age  Replacement  Policy  Algorithm,  using 
a closed  form  expected  long  term  average  cost  per  unit  time  expression 
and  a selective  search  technique  to  find  the  minimum  cost  policy.  This 
listing  is  for  a maximum  size  problem  of  thirty  states  and  twenty  poss- 
ible discrete  holding  time  units  per  state.  For  larger  problems  or  to 
save  array  space  in  smaller  problems,  the  user  may  modify  the  DIMENSION 
i and  COMMON  statements  to  the  appropriate  size  without  adverse  effect 

on  the  routine  execution.  All  output  is  internally  coded  in  the  sub- 
routine  so  that  it  may  be  used  without  the  main  program  found  in  Appen- 
^ dix  A if  the  user  supplies  a suitable  main  program  to  input  the  para- 

meters of  the  problem. 


1 

1 


35 


• 

• 

• 

« 

• 

A 

• 

• 

Z 

• 

« 

• 

• 

o 

« 

• 

M 

« 

% 

• 

z « 

« 

< 

z « 

• 

a 

lU  z « 

• 

o 

K 3 « 

• 

« 

M 

z « 

9 

« 

a 

o •-  « 

• 

UJ 

z z « 

e 

« 

t- 

O M « 

N 

« 

lii 

-1  z • 

• 

« 

o 

• 

e 

• 

hi  lu  « 

n 

• 

> 

o z • 

• 

f 

o 

< K « 

e 

i 

X 

z » 

lA 

• 

a 

hi  z • 

90 

• 

< 

> o # 

X 

• 

z 

< IL  • 

a 

• 

1 

• 

• 

• 

O 111  • 

« 

z 

hi  Z • 

o 

• 

III 

K 3 « 

lA 

• 

tf) 

u fi  • 

90 

« 

hi  hi  • 

z 

• 

< 

aw  • 

¥~ 

« 

X o * 

• 

« 

et 

111  z » 

09 

• 

o 

a « 

O 

« 

h. 

z « 

lA 

« 

o z • 

90 

• 

► 

h.  w « 

o 

• 

o 

z • 

• 

• 

M 

Z X « 

09 

• 

al 

o hi  « 

e 

« 

O 

M «>  • 

lA 

• 

a 

tfl  • 

90  O 

• 

(A  hi  « 

Z N 

• 

t 

hi  > « 

» • 

• 

z 

z •-  • 

<•  O 

« 

III 

a K • 

e n 

1 

« 

z 

X w • 

lA  w 

« 

III 

hi  hi  • 

^ z 

« 

u 

w « 

X ► 

• 

< 

z hi  • 

• z • 

« 

J 

z w • 

09  90  09 

• 

a 

o • 

0*0 

• 

hi  h.  X « 

hi  J «A 

• 

a 

« 

0 » 90 

• 

O Q • 

o >•  z 

• 

J 

hi  z • 

n a z « 

• 

« 

w « • 

w j • e 

• 

z 

o • 

X • -»  « 

• 

M 

j hi  • 

• «*  e w > 

• 

t- 

w z • 

« Z N hi  w 

z • 

& 

. • 

O ^ > M 

^ * 

o 

X ► # 

n • M X J 

i • 

• 

• ••  Z lA  p • 

• 

z 

V t~  • 

e o a X 6 

* 

W * 

5 

Z M • 

W lA  90 

z • 

u. 

•*  z • 

«r  «'  Z Z J 

M # 

M 5 • 

a X o o X II  < 

h • 

hi 

3 • 

0 m 09  0m 

3 • 

z 

Z • 

Z «»  M M »>  ».  < 

• 

6 9 

M 

lA  hi  • 

B O Z Z M z 1 

fi  • 

R C • 

1 fi  hi  u Z X ! 

fi  t 

2 

HI  • 

1 w f 1 M H 4 

5 # 

6 

UK  • 

0 X M hi  1 

• 

¥>  • 

z 

p w • 

w X e o h-  M I 

• 

« 

z o • 

X hi 

• 

• 

3 

a u • 

M 

O .i  Ml 

M • K 


► 

111  o 

U 

K O 

M 

X X 

J 

t- 

O 

(A  Z 

a 

W 

z X 

w 

M 111 

X 

•« 

z z 

»- 

09 

o o 

90 

90 

M h. 

Z 

9 

lA 

M 

90 

M O 

• 

W Z 

Z 

•» 

hi  3 

o 

90 

O fi 

IL 

X 

o 

a 

h’ 

lA 

(A  Z 

hi 

♦ 

o hi 

Z 

U 0. 

M 

a 

1- 

09 

Z 3 

•» 

3 - 

z 

Z l>  z 

z 

X 

00  U J 

3 

0 

z w • 

O 

") 

M hi  H 

e 

X 00 

Z Z Z 

o •• 

• 

«•  o z 

X 

lA  Z 

o 

. Z K 

Z • 1- 

J 

1 •« 

O Z 4A 

z 

N 

N 

h.  o M 

X • 

A • 

»• 

K s « «»  ^ 

*•*  N*yKNn 

•i  A ^ III  w w 

•^M—^—IIHIlillwji 

H • w < • • 

S^S-^^P  •C  w 

'"T;i9*>«09SXCS 


X K j 
< 
- u 


8?5 


Z »>  H 
U < 

« a u 

< hi  u 

M HI  H ^ 
wt  <« 

N n 
Z K 

X M M Q 

M z 5 

HI  •* 
{•H. 


U M M X J 

u « <#  e 

^ ♦ I < UJ  • 

Z > M 

I «n  M « 

^ ^ hi  M N 

J »>  X 

N N X * 

" - J H - 

z z 9 w 

«l  W M U X z 

J ^ J J X z 

< 

u 


U U U Cl  u u u 


3 COMTINUE 

CHECK  EOtt  A CHANGE  OF  POLICY  DURING  THIS  ITERATION 
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